Method for Generating Splines Based on Surface Intersection Constraints in a Computer Image Generation System

ABSTRACT

A representation of a surface in a three-dimensional space is obtained. A first input representing a starting point and a second input representing a next point are obtained. A representation of a surface-aware spline comprising vertices is generated, with the representation of the surface-aware spline including a starting vertex corresponding to the starting point and a next vertex corresponding to the next point. First and second projection points corresponding to projections of a first vertex and a second vertex onto the surface are determined. New points corresponding to equal distance points for the first and second vertices aligned with the first and second projection points are determined, and a rigid transformation is determined from the new points. The representation of the surface-aware spline is adjusted based on a transformation of the new points using the rigid transformation.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Non-Provisional patentapplication Ser. No. 17/098,198, filed Nov. 13, 2020, which claimspriority to and the benefit of U.S. Provisional Patent Application No.63/032,409, filed May 29, 2020, which are incorporated herein byreference in their entirety.

FIELD

The present disclosure generally relates to computer-based methods ofprocedurally generating representations of three-dimensional splines andmore particularly to techniques and systems for generatingrepresentations of fibers in a virtual space that are constrained fromintersecting a virtual surface in the virtual space and usable for imagegeneration.

BACKGROUND

As digital animation in movies and games has increased in popularity, sohas the complexity of the models and the virtual environments in whichthey interact. Various tools (e.g., physics engines) have been developedto create realistic interactions between objects. However, simulatingrealistic fibers (such as hair, fur, feathers, etc.) is a challenge andlimitations of available computing power and other considerations oftenled to undesirable tradeoffs between realism in imagery and limiting useof computational resources.

For example, simulating hair may involve complex differential equationsthat greatly increase computational resource consumption the longer andmore numerous the hairs are. Furthermore, if a digital artist needs tomake changes to such simulated hair, the process can be laborious,time-consuming, and involve much experimentation and trial and error tomanipulate the hair into a proper or visually appealing position.

SUMMARY

Techniques and systems described below relate to generating digitalfibers in a virtual space that are constrained from intersecting avirtual surface in the virtual space. In one example, a representationof a curved surface in a three-dimensional space is obtained. In theexample, a first input representing a position of a starting point of asurface-aware spline that is on, or relative to, the curved surface isobtained. In the example, a second input representing a next point ofthe surface-aware spline is obtained. Still in the example,representation of the surface-aware spline comprising N vertices isgenerated, including a first vertex and a second vertex, with therepresentation of the surface-aware spline including a starting vertexcorresponding to the starting point and a next vertex corresponding tothe next point. Still further in the example, first projection pointcorresponding to a first projection of the first vertex onto the curvedsurface is determined.

Further in the example, a second projection point corresponding to asecond projection of the second vertex onto the curved surface isdetermined. Still in the example, new points corresponding to equaldistance points for the first and second vertices aligned with the firstand second projection points are determined. A bending rigidtransformation might be determined from the new points. Therepresentation of the surface-aware spline might be adjusted based on atransformation of the new points using the bending rigid transformation.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments in accordance with the present disclosure will bedescribed with reference to the drawings, in which:

FIGS. 1A-1C are diagrams that illustrate procedurally simulating a fiberin accordance with an embodiment.

FIGS. 2A-2C are diagrams that illustrate unbending a curve in accordancewith an embodiment.

FIG. 3 is a flowchart that illustrates an example of procedurallysimulating fiber in accordance with an embodiment.

FIG. 4 is a system for generating and placing one or moresurface-constrained splines within a computer animation in accordancewith an embodiment.

FIG. 5 illustrates an example visual content generation system as mightbe used to generate imagery in the form of still images and/or videosequences of images.

FIG. 6 is a block diagram illustrating an example computer system uponwhich computer systems of the systems illustrated in FIGS. 1 and 5 maybe implemented.

DETAILED DESCRIPTION

In the following description, various embodiments will be described. Forpurposes of explanation, specific configurations and details are setforth in order to provide a thorough understanding of the embodiments.However, it will also be apparent to one skilled in the art that theembodiments may be practiced without the specific details. Furthermore,well-known features may be omitted or simplified in order not to obscurethe embodiment being described.

In some examples, objects can be represented in computer memory by adata structure defining a mathematical representation of atwo-dimensional (2D) or three-dimensional (3D) physical item or being.The representation might be stored as a mesh of vertices defining atessellated manifold of a boundary for the object, and a texture forcoloring that boundary. In some examples, a mesh can be represented inmemory as data structures referring to a collection of vertices, edges,and/or faces that defines the shape of a polyhedral object in computergraphics modeling. In some examples, a scene can be represented as datastructures referring to a virtual workspace for editing and/or renderinga three-dimensional project. In some examples, an interface might becomputer hardware or software designed to communicate informationbetween hardware devices, between software programs, between devices andprograms, or between a device and a user.

In an example computer image generation process, a digital artist orother image creator might want to generate an image including objects,such as a head, a person, a box, etc. and fibers (e.g., hair, rope,braid, etc.) wherein the number of fibers is high enough that it mightbe impractical to have the creator manually specify the positions ofeach fiber, while avoiding visual artifacts such as fibers intersectingsurfaces or intersecting themselves. In some embodiments, it might bedesirable to have a computer process place the fibers according to someprocedural constraints, which might be specified by the creator using acomputer interface. For example, a fiber might be represented as apiecewise linear curve or a parameterized curve, such as a cubic curve,a Bezier curve, a line segment, etc. and “placing” that fiber into ascene with other objects might be done by specifying parameters of thecurve. For a line segment, it might be sufficient to specify two endpoints and for a cubic spline, it might be sufficient to specify two endpoints and a tangent vector at each end point.

FIGS. 1A-1C illustrate procedural simulations 100 a, 100 b, and 100 c ofhair or another fiber in accordance with of an embodiment of the presentdisclosure. Specifically,

FIG. 1A depicts a spline 104 that has been generated to represent afiber on a digital model 102 of a head starting from start point 106 andending at end point 108. Cursor 112 may be used to modify one or moreparameters of spline 104, such as a velocity associated with the splineor to move start point 106 or end point 108. Dashed line 110 illustratea spline without a constraint preventing the spline from intersecting asurface of digital model 102, whereas, as can be seen in FIG. 1, spline104 is constrained in a manner described in the present disclosure suchthat it does not intersect with the surface of the model except possiblyat starting point 106.

Procedural simulation 100 a depicted in FIG. 1A may be produced byexecution of software on one or more computer systems (e.g., each like acomputer system 400 illustrated in FIG. 4). A visual content generationsystem 500 (see FIG. 5) may be configured to receive the values of theone or more parameters as input and output a simulation, such asprocedural simulation 100 a, as one or more static images and/or one ormore animated videos. For example, data corresponding to proceduralsimulation 100 a may be supplied to an animation creation system 530,which may output procedural simulation 100 a (such as to an animationsequence storage 538, a scene description storage 536, an object storage534, and/or a data store 532).

Digital model 102 may be visual representation of a two-dimensional orthree-dimensional object. For illustrative purposes only, digital model102 of FIG. 1A is depicted as a human head. Digital model 102 may becomprised of multiple two-dimensional polygons, and the plane regions ofsuch polygons may comprise the surface (e.g., skin) of digital model102. In various embodiments, the surface of digital model 102 may berepresented in memory as a constraint on spline 104, in which case, aprogram that processes the surface might take the surface of digitalmodel 102 into account to ensure that spline 104 passes around or lieson top of or above the surface, but does not go through the surface. Inthis manner, techniques of the present disclosure may be utilized togenerate realistic hair and/or other flexible fibers that appear toreact to the surfaces of models as if they were solid. The process mightalso deal with splines that are automatically placed so that not only dothey not intersect a given surface and avoid a negative distance abovethe surface, but they flow with a particular positive and nonzerodistance from a surface. For example, one spline might be automaticallyplaced so that it originates at a point on a surface with a particularinitial tangent, but also is positioned so that much of the spline ispositioned some distance from the surface, as might be represented by ageodesic of the surface.

Spline 104 may be a polynomial curve in 2D or 3D space. In someembodiments, spline 104 is a cubic spline comprising a plurality ofsegments between points on the spline, where each segment is athird-degree polynomial. Further, the cubic spline may be a cubic splineusable to interpolate data points into pixels for display. Spline 104may begin at start point 106 and end at end point 108 or vice versa. Thecurvature of spline 104 may be specified by inputs form a user (e.g.,through manipulation of cursor 112).

Start point 106 may be a point from which generation of spline 104begins. In some embodiments, start point 106 is also associated with astart point tangent plane (not shown) that runs through start point 106.An orientation of the start point tangent plane may affect the curvatureof spline 104. In some embodiments, a velocity value at the start pointtangent plane may likewise affect the curvature of spline 104. Forexample, the velocity value may cause the curve to move farther from thesurface, but the curvature of spline 104 will still follow the naturalcurvature of the surface. In some embodiments, the orientation of thestart point tangent plane and/or velocity can be controlled by the usermanipulating cursor 112, such as via input device 614 or cursor control616 of FIG. 4. Additionally or alternatively, in some embodiments theorientation of the start point tangent plane may be affected by thesurface of digital model 102; for example, the start point tangent planeat start point 106 may be based on a normal to a curve of the surface ofdigital model 102 at start point 106.

End point 108 may be a point at which generation of spline 104 ends. Insome embodiments, end point 108 is also associated with an end pointtangent plane (not shown) different from the start point tangent plane.An orientation of the end point tangent plane may likewise affect thecurvature of spline 104.

Dashed line 110 may be a line that indicates the path that spline 104would have followed had it not been prevented from intersecting digitalmodel 102 in accordance with the techniques of the present disclosure.In some implementations, dashed line 110 may be shown via an interfaceon a client device as a reference to assist a digital artist. In otherimplementations, dashed line 110 may not be present.

Cursor 112 may be a graphical indicator used to show a current positionfor user interaction on a graphical display device. The position and/ororientation of cursor 112 may be controlled by a user via an inputdevice such as input device 614 or cursor control 616 of FIG. 4. Changein position or orientation of cursor 112 may affect characteristics ofspline 104 (e.g., changing a tangent value, changing a velocity, movingthe position of end point 108 or start point 106, etc.), which mayaffect the path followed by the curve of spline 104.

FIG. 1B illustrates a procedural simulation 100 b depicting a 3D model120 rendered by a computer animation system, such as computer system400. As with FIG. 1A, visual content generation system 500 may beconfigured to receive the values of the one or more parameters as inputand output a simulation, such as procedural simulation 100 b, as one ormore static images and/or one or more animated videos. For example, datacorresponding to procedural simulation 100 b may be supplied toanimation creation system 530, which may output procedural simulation100 b (such as to animation sequence storage 538, scene descriptionstorage 536, object storage 534, and/or data store 532).

3D model 120 of FIG. 1B includes a surface 122 having surfacedimensions, curvature, and the like, which define 3D model 120 and areused to constrain one or more splines generated to represent fibers over3D model 120. Procedural simulation 100 b of FIG. 1B further depicts asurface-constrained spline 128 that has been generated to represent thefiber on 3D model 120 from a start point 124 to an end point 126. Anunconstrained spline 130 illustrates a spline without a constraintpreventing the spline from intersecting a surface of 3D model 120, suchas surface 122, whereas, as can be seen in FIG. 1, surface-constrainedspline 128 is constrained in a manner described in the presentdisclosure such that it does not intersect with surface 122 of 3D model120.

When modeling surface-constrained spline 128, initially unconstrainedspline 130 may be used to define at least a portion the shape of thecorresponding underlying structure, such as by the polynomials or otherfunctions that make up unconstrained spline 130 (e.g., a cubic spline).However, when manipulating unconstrained spline 130 in proceduralsimulation 100 b, changing a point along unconstrained spline 130,including the start or end point of unconstrained spline 130, may causeunconstrained spline 130 to intersect one or more points or areas alongsurface 122 of 3D model 120. This may be caused when the changed oradjusted points are nearby a surface, on opposite surfaces of 3D model120 (e.g., an inner and outer surface), or the like. For example, whenunconstrained spline 130 travels from start point 124 to end point 126,unconstrained spline travels through 3D model 120 (not shown) and exitsas shown in the dashed line nearby end point 126. In such examples,unconstrained spline 130 may not be able to properly represent the fibertravelling over or nearby (e.g., directly on the surface or at adistance from the surface). In order to properly represent such a fiberusing unconstrained spline 130, multiple additional points (e.g., tens,hundreds, or thousands) may be required to be placed at or over surface122 in order to cause the functions representing unconstrained spline130 to properly represent such a fiber without intersecting surface 122.This causes computational disadvantages by requiring additional data,points, and functions to represent the spline, as well as requiringsignificant time and effort to select the proper spline points.

Thus, placing a fiber that is meant to wrap from a point on the innercircular surface to the outer circular surface, would causeunconstrained spline 130 to intersect surface 122. In order to properlyrepresent such a fiber, surface-constrained spline 128 further usespoints over the surface and projected on to the surface, with tangentplanes of these projected points, to represent a further discreetfunction that allows surface-constrained spline 128 to be placed on orover surface 122. A user, such as an animator, may adjust start point124 and end point 126 to cause the fiber represented assurface-constrained spline 128 to move in different directions and todifferent placements over surface 122. A tool 132 in proceduralsimulation 100 b may be controlled by a user in order to select pointsalong surface-constrained spline 128, such as start point 124, asubsequent or next point along surface-constrained spline 128, and/orend point 126, as well as tangents and tangent planes at these points.For example, surface-constrained spline 128 is shown with a curvaturenear end point 126 along or nearby surface 122, which may be made bycontrolling a point prior to the end point, as well as the tangentplanes defined at these points. This may include assigning points ofattraction that cause the curvature of surface-constrained spline 128 tobend in particular directions. Further the overall curvature, includingdirection, orientation, and the like, may be controlled using thesetangent planes at these points and velocities of curvature along thespline.

FIG. 1C includes procedural simulation 100 c depicting 3D model 120(shown in procedural simulation 100 b of FIG. 1B) rendered by a computeranimation system, such as computer system 400. As with FIGS. 1A and 1B,visual content generation system 500 may be configured to receive thevalues of the one or more parameters as input and output a simulation,such as procedural simulation 100 c, as one or more static images and/orone or more animated videos. For example, data corresponding toprocedural simulation 100 c may be supplied to animation creation system530, which may output procedural simulation 100 c (such as to animationsequence storage 538, scene description storage 536, object storage 534,and/or data store 532).

3D model 120 in procedural simulation 100 c is similarly shown withsurface 122 having surface dimensions, curvature, and the like, whichdefine 3D model 120 and are used to constrain one or more splinesgenerated to represent fibers over 3D model 120. Procedural simulation100 c of FIG. 1C shows a representation of a surface-constrained spline138 having a start point 134 and an end point 136, where in contrast toFIG. 2B, end point 136 is not placed directly on surface 122 andtherefore surface-constrained spline may not travel directly oversurface 122. In order to define at least a portion of the functions,curvature, and/or points of or along surface-constrained spline 138, anunconstrained spline 140, such as a cubic spline, is shown in proceduralsimulation 100 c. Unconstrained spline 140 may start at the same orsimilar start point 134 of surface-constrained spline 138 and travel toor nearby end point 136, intersecting or being curved to or nearby anypoints of interest or attraction, which defines the function(s) ofunconstrained spline 140 (e.g., the polynomials or other functions meantto represent unconstrained spline 140). However, as shown in proceduralsimulation 100 c, unconstrained spline 140 intersects surface 122 whentraveling from start point 134 to end point 136, shown as the dashedline exiting 3D model 120. Thus, surface-constrained spline 138 may beestablished having functions and equations that define the curvature,direction, and placement of surface-constrained spline 138 using thesurface dimensions, curvature, and the like of surface 122, therebypreventing surface intersection.

A tool 142 may be used to move end point 136 away and apart from surface122 so that surface-constrained spline 138 may include an area under acurve 144 that is not located directly on surface 122, which may changethe calculations and curvature of surface-constrained spline 138 (e.g.,the function representing surface-constrained spline 138 that considerssurface 122). Tool 142 may be used to adjust the points along and/or atthe ends of surface-constrained spline 138, which may include removal ofsuch points from surface 122. This may occur while surface-constrainedspline 138 is still constrained by surface 122 to prevent anyintersections with surface 122. A change in the points may also includea change in tangent planes and/or velocities at points alongsurface-constrained spline 138, which may be provided through tool 142.Further, tool 142 may be used to place or adjust points of attraction tocause changes in the curvature, angle, placement, or orientation ofsurface-constrained spline 138, such as by moving portions or all ofsurface-constrained spline 138 towards or to intersect one or more ofthese points of attraction. Calculation of the curvature of spline 104and surface-constrained splines 128 and 138 in FIGS. 1A-C may beperformed as discussed herein, for example, using the calculations andequations discussed in reference to FIGS. 2A-C.

FIGS. 2A-2C illustrate examples 200 of an embodiment of the presentdisclosure. Specifically, FIGS. 2A-2C illustrate an estimation of arigid transformation M_(k) that cancels the bending component betweenconsecutive curve samples x_(k) and x_(k+1) in accordance with anembodiment. In FIGS. 2A-2C, a surface

is intended to represent a curve of surface of a model; for example,surface

may be a portion of the surface of digital model 102 of FIG. 1. Notethat the elements depicted in FIGS. 2A-2C are shown in a two-dimensionalcross-section of a 3D space for ease of illustration. Techniques of thepresent disclosure may be operable to compute a smooth parametric curve,

, between two points A and B on the same side as surface

such that the derivative of the curve

at the point A is V_(A), the derivative of the curve

at the point B is V_(B), and the curve

follows the topology of surface

without intersecting surface

except possibly at a starting point, x₁, (not shown) of the curve

.

In an embodiment, the curve

may be represented in memory and processing by a succession of N samplepoints (e.g., x₁, x₂, . . . , x_(N)). In some embodiments, number ofsamples N may be a user-defined parameter. The concatenation of allsamples points in a single vector may be denoted by X in Equation 1.

$\begin{matrix}{X = \begin{pmatrix}x_{1} \\x_{2} \\\vdots \\x_{N}\end{pmatrix}} & \left( {{Eqn}.\mspace{14mu} 1} \right)\end{matrix}$

In the discrete domain, the constraints at the end points of the curve

may be expressed as in Equation 2.

$\begin{matrix}\begin{matrix}A & {= x_{1}} \\B & {= x_{2}} \\V_{A} & {= \frac{{{- 3}x_{1}} + {4x_{2}} - x_{3}}{2N}} \\V_{B} & {= \frac{x_{N - 2} - {4x_{N - 1}} + {3x_{N}}}{2N}}\end{matrix} & \left( {{Eqn}.\mspace{14mu} 2} \right)\end{matrix}$

In a case without surface

, such a smooth curve may be obtained by solving Equation 3, where L isa linear operator subject to Equation 2 that evaluates to an(approximate) second derivative at each sample as in Equation 4.

:

$\begin{matrix}{{L^{2}(X)} = 0} & \left( {{Eqn}.\mspace{14mu} 3} \right) \\{L = \begin{pmatrix}1 & {- 2} & 1 & 0 & 0 & \; \\0 & 1 & {- 2} & 1 & 0 & \ldots \\0 & 0 & 1 & {- 2} & 1 & \; \\\mspace{11mu} & \; & \vdots & \; & \; & \ddots\end{pmatrix}} & \left( {{Eqn}.\mspace{14mu} 4} \right)\end{matrix}$

A technique of the present disclosure is to modify operator L so that itcomputes a second derivative that takes into account the local curvatureof surface

in the vicinity of each sample. For example, if three consecutivesamples x_(k−1), x_(k), x_(k+1) are equally spaced on a geodesic path ofsurface

(e.g., a path of shortest length), then operator L should evaluate to 0at x_(k).

In some implementations, an automatic spline generator might initializea seeking process using an initial curve, X⁰, that joins A to B. Theinitial curve, X⁰, need not be constrained to respect all theconstraints of Equation 2, though convergence may be affected if thisinitial curve is positioned too far from the optimal solution. In someembodiments, an approach is to start from an approximate geodesic pathon surface

that joins projections, P_(A) and P_(B), of A and B onto surface

.

The automatic spline generator might then iteratively refine the curveas follows:

-   -   1. Compute P^(i)=p₂, . . . , p_(N)), which is the projection of        the current curve X^(i) onto surface        .    -   2. Construct a discrete second derivative that compensates for        the curvature of surface        at each point p_(k), referred to in the present disclosure as        new curvature-compensated operator L_(c) _(i) . L_(c) _(i) is        indexed by i because L_(c) is updated at each step; that is,        with each new curve the system of the present disclosure        computes a new set of transformations M_(k) (see below) and form        a new operator L_(c).    -   3. Compute a new curve X^(i+1) solution of the        curvature-compensated operator L_(c) _(i) ² (X^(i))=0 subject to        (1).    -   4. Check for convergence: if ∥X^(i+1)−X^(i)∥<∈ (where ∈ may be a        pre-defined threshold) then terminate the algorithm. Otherwise        return to 1.

FIGS. 2A-2C illustrate techniques for an automatic curve generator togenerate a curve corresponding to a local bending of a surface

between consecutive curve points using rigid transformations(rotations).

In FIG. 2A, x_(k) and x_(k+1) represent a pair of adjacent curve samples(e.g., points in 3D space), and p_(k) and p_(k+1) represent theirprojection onto surface

, with d_(k) being a distance from p_(k) to x_(k), and d_(k+1) being adistance from p_(k+1) to x_(k+1). In embodiments, p_(k) and p_(k+1) arepoints on surface

that are closest to their respective x_(k) and x_(k+1). If surface

is smooth, the normal to surface

at p_(k) will point to x_(k) and the normal to surface

at p_(k+1) will point to x_(k+1); for example, if surface

is a plane, then p_(k) is the orthogonal projection of x_(k) ontosurface

. Various techniques may be employed to find the closest p_(k) to x_(k)on surface

, such as, for example, by partitioning surface

using tree structures (e.g., bounding volume hierarchy, k-d tree, etc.)in order to quickly find a first candidate, then refining the solutionwith iterations using algorithms such as a Gauss-Newton algorithm.

Often, d_(k)≠d_(k+1), which may present a challenge in estimating thelocal bending at the x_(k) because the displacement step from x_(k) tox_(k+1) is not tangential only. It may be easier to estimate the bendingat p_(k) but defining the operator L based on the bending at p_(k) mayyield inaccurate results if the p_(k) and x_(k) are too far apart.Instead, in FIG. 2B, the system of the present disclosure may beconfigured to compute new points a_(k) and b_(k) on segments[p_(k)x_(k)] and [p_(k+1)x_(k+1)] such that a_(k) and b_(k) are at thesame distance r_(k) from surface

by computing r_(k) as the average of d_(k) and d_(k+1) as in Equation 5so that a_(k) and b_(k) are close to the initial curve points in FIG.2B. The bending from a_(k) and b_(k) will thus work as a goodapproximation of the bending between x_(k) and x_(k+1) in Equation 5.

$\begin{matrix}{r_{k} = \frac{d_{k}d_{k + 1}}{2}} & \left( {{Eqn}.\mspace{14mu} 5} \right)\end{matrix}$

In FIG. 2C, the system determines plane

_(a) _(k) , a plane that goes through point a_(k) and whose normalvector is given by x_(k)−p_(k). Similarly, the system determines plane

_(b) _(k) , a plane that goes through point b_(k) and whose normalvector is given by x_(k+1)−p_(k+1) in FIG. 2C. In the absence of bending(e.g., when surface

is planar),

_(a) _(k) and

_(b) _(k) would be same plane. Consequently, in most cases the system ofthe present disclosure can compensate for the bending at a_(k) bycomputing an appropriate transformation M_(k) that projects plane

_(b) _(k) onto plane

_(a) _(k) . In some embodiments of the present disclosure, such atransformation may be performed by adding the constraints such thatM_(k) is a rigid transformation (no scaling) and M_(k) minimizes∥b_(k)′−b_(k)∥²+∥a_(k)′−a_(k)∥² where b_(k)′=M_(k)(b_(k)) anda_(k)′=M_(k) ⁻¹(a_(k)). The second constraint implies that M_(k) isconfigured to preserve the position of a_(k) and b_(k) within athreshold, ideally as much as possible. Note that since M_(k) projects

_(b) _(k) onto

_(a) _(k) , its inverse M_(k) ⁻¹ projects the plane

_(a) _(k) onto the plane

_(b) _(k) .

The first constraint allows M_(k) to be in matrix form. For any point pon surface

, Equation 6 would be satisfied.

M _(k)(p)=R _(k) ·p+T _(k)  (Eqn. 6)

In Equation 6, R_(k) is a 3×3 rotation matrix and T_(k) is a translationvector. Thus, Equation 7 would represent such operations.

M _(k) ⁻¹(p)=R _(k) ^(T) ·p−R _(k) ^(T) T _(k)  (Eqn. 7)

In this manner, M_(k) is a rigid transformation (3×3 rotation andtranslation) that transforms the plane

_(a) _(k) into the plane

_(b) _(k) , wherein every point of

_(b) _(k) should land on plane

_(a) _(k) . The second constraint provides for canceling the effect ofthe bending of surface

while avoiding twisting by constraining a_(k) and b_(k) from movingwithin a threshold, which could prevent a_(k) and b_(k) from moving anymore than necessary. Using the transformations M_(k), the system of thepresent disclosure may be configured to derive the expression ofcurvature-compensated second derivatives (Δ) using finite differencesshown in Equations 8-9, which allows the curvature-compensated operatorL_(c) of the present disclosure to be expressed in matrix form as inEquation 10.

Δ(x _(k))=M _(k−1) ⁻¹(x _(k−1))−2x _(k) +M _(k)(x _(k+1))  (Eqn. 8)

$\begin{matrix}{\mspace{79mu}{= {{R_{k - 1}^{T} \cdot x_{k - 1}} - {2x_{k}} + {R_{k} \cdot x_{k + 1}} + T_{k} - {R_{k - 1}^{T}T_{k - 1}}}}} & \left( {{Eqn}.\mspace{14mu} 9} \right) \\{{L_{c}(X)} = {{\begin{pmatrix}R_{1}^{T} & {- 2} & R_{2} & 0 & 0 & \; \\0 & R_{2}^{T} & {- 2} & R_{3} & 0 & \ldots \\0 & 0 & R_{3}^{T} & {- 2} & R_{4} & \; \\\; & \; & \vdots & \; & \; & \ddots\end{pmatrix}\begin{pmatrix}x_{1} \\x_{2} \\\vdots \\x_{N}\end{pmatrix}} + \begin{pmatrix}{T_{2} - {R_{1}^{T}T_{1}}} \\{T_{3} - {R_{2}^{T}T_{2}}} \\\vdots \\{T_{N} - {R_{N - 1}^{T}T_{N - 1}}}\end{pmatrix}}} & \left( {{Eqn}.\mspace{14mu} 10} \right)\end{matrix}$

Thus, X may be computed by solving Equation 11 for X. In this manner,the curvature-compensated operator L_(c) illustrated by Equation 11allows the computation of “surface-aware” splines in an efficient way.

L _(c) ² X)=0  (Eqn. 11)

The curvature-compensated operator L_(c) may be used to compute paths(e.g., geodesic paths) on surfaces, to compute splines embedded in asurface, and/or to compute Riemannian surface-aware splines thatautomatically bend in the vicinity of a surface while following thecurvature of the surface. Whereas computation of an embedding of asurface mesh in a high dimensional space may take minutes to precomputegeodesic paths and splines on surfaces using weighted averages, with thetechnique described above there is no such overhead and the algorithmmay be run quickly in real time.

The energy function of the system may present multiple local minimas,especially if there are concavities in the surface. In order toguarantee convergence, the system of the present disclosure ensures thateach new curve X^(i+1) brings an improvement over the previous solutionXL by comparing their respective Laplacian energy as illustrated byEquation 12.

∥L _(c) _(i+1) (X ^(i+1))∥² <∥L _(c) _(i) (X ¹)∥²  (Eqn. 12)

If Equation 12 is not satisfied, the system iteratively updates X^(i+1)as shown in Equation. 13.

$\begin{matrix}\left. X^{i + 1}\leftarrow\frac{X^{i} + X^{i + 1}}{2} \right. & \left( {{Eqn}.\mspace{14mu} 13} \right)\end{matrix}$

In this manner, the updated X^(i+1) will always satisfy Equation 12 atsome point because X^(i)X^(i+1) is a descent direction for the Laplacianenergy.

Equation, 11 may be a simple sparse/banded linear system because thematrix in L_(c) is block tridiagonal. Thus, it can be solved in lineartime (e.g., by computing its Cholesky factorization). This might allowfor more efficient use of limited computational resources inprocedurally placing objects in a virtual space.

FIG. 3 is a flowchart illustrating an example of a process 300 forprocedurally generating a spline in accordance with various embodiments.Some or all of the process 300 (or any other processes described, orvariations and/or combinations of those processes) may be performedunder the control of one or more computer systems configured withexecutable instructions and/or other data, and may be implemented asexecutable instructions executing collectively on one or moreprocessors. The executable instructions and/or other data may be storedon a non-transitory computer-readable storage medium (e.g., a computerprogram persistently stored on magnetic, optical, or flash media).

For example, some or all of process 300 may be performed by any suitablesystem, such as by an application server, by multiple computing devicesin a distributed system of a computing resource service provider, or byany electronic computing device such as computer system 400 of FIG. 4,which may be a member of an animation creation system, such as theanimation system 530 component of the of visual content generationsystem 500 depicted in FIG. 5. The process 300 includes a series ofoperations wherein a set of constraints for generation of a cubic splineare obtained and, for each of the points except the two outer points ona surface that the curve is to follow but not intersect, a rigidtransformation is performed on the preceding point and the succeedingpoint. Then, the cubic spline may be computed based on an operationperformed to a matrix of the points and the preceding and succeedingpoint transformations.

In step 302, the system performing the process 300 obtains a set ofconstraints for the generation of the cubic spline. The set ofconstraints may include a set of points in a virtual space approximatingcurve corresponding to a surface that the cubic spline is constrainednot to intersect. In some embodiments, the set of constraints mayinclude a velocity value.

In step 304, the system performing the process 300 may begin iteratingthrough the set of points. Because the rigid transformation performed instep 308 is performed on a preceding point to the current point, thesystem may start the iteration with the succeeding/next point after thestart point of the set of points.

In step 306, the system performing the process 300 may determine atangent plane to the surface at the point.

In step 308, the system performing the process 300 may perform a rigidtransformation on the preceding point to the point being processed todetermine a first intersecting point corresponding to the precedingpoint that intersects with the tangent plane.

In step 310, the system performing the process 300 may perform a rigidtransformation on the succeeding point to the point being processed todetermine a second intersecting point corresponding to the succeedingpoint that intersects with the tangent plane.

In step 312, the system performing the process 300 may determine whetherit has iterated through all of the points, and if not, returns to 304 toperform the steps of 304-312 against the next point in the set ofpoints. Because the rigid transformation in step 310 is performed on asucceeding point to the current point, the iterating can end at thesecond to the last point in the set of points.

The system performing the process 300 may retain the first and secondintersecting points as steps 306-312 are performed for each of thepoints, thereby producing sets of first and second intersecting points.In step 314, the system performing the process 300 may generate a matrixbased on the points and the sets of first and second intersectingpoints, similar to the matrix described in conjunction with FIG. 2.

Lastly, in step 316, the system performing the process 300 may, solvethe resultant matrix from the start point to the end point to producethe cubic spline. The cubic spline may subsequently be rendered as afiber (e.g., a hair, a rope, a thread, etc.) or may serve as a centerline guide for generating a more complex flexible object (e.g., a braid,a snake, a hose, etc.). Note that one or more of the operationsperformed in steps 302-316 may be performed in various orders andcombinations, including in parallel.

Steps 302-316 describe one possible implementation of techniques of thepresent disclosure. It is contemplated that other implementations arepossible. For example, in an alternative implementation, for each pointx_(k) find its closest point p_(k) on the surface S. Then, for each pairof consecutive points [x_(k), x_(k+1)], compute a rigid transformationM_(k) (R_(k), T_(k)) according to x_(k), x_(k+1), p_(k) and p_(k+1) asdescribed above in conjunction with FIGS. 2A-2C.

FIG. 4 illustrates a system for generating and placing one or moresurface-constrained splines within a computer animation, in anembodiment. The system includes a surface and spline dataset 402, aspline processing unit 406, a renderer 418, a user interface (UI) 420,and spline input data 422.

A user 440 may interact with the UI 420 to define one or more splinesthat is to be constrained by a surface such that the spline does notintersect the surface when one or more points or vertices for the splineare designated and/or moved. This spline may therefore correspond to afiber that is used to be placed relative to a surface. Spline input data422 may indicate, for example, the criteria for the spline, such as thecurvature, polynomials, placement, vertices, and the like of the spline.Surface and spline dataset 402 may store data for surfaces used toconstrain splines, such as characters, creatures, or objects. Splinedataset 402 may be loaded with data from a source of an animation, suchas a tessellated mesh, subdivision surface, or the like used to define acharacter, creature, or object. Spline processing unit 406 may utilizethe methods and processes described herein to take spline input data 422and constrain the corresponding spline according to the surface utilizedwith the spline from spline dataset 402. The spline processing unit 406may constrain the spline using the points and vertices of the spline(e.g., data defining a cubic spline or the like) with the correspondingtangent planes to the surface, as described herein.

Spline processing unit 406 includes a processor 410 that executesprogram code 412 to constrain splines designated in spline input data422 using a surface 414 from surface and spline dataset 402. Splineprocessing unit 406 may further store surface and spline data 408 todataset 402 so that the corresponding surface and fibers represented bythe splines may be rendered by renderer 418 for a realistic a scenevisualization using fibers corresponding to the splines. For example,spline processing unit 406 may initiate the process by taking splineinput data 422 with surface 414 and constraining the spline by surface414 so that a surface-constrained spline 416 does not intersect surface414. Based on surface-constrained spline 416, spline processing unit 406may then output surface and spline data 408, which may include splineparameters 404 stored by dataset 402 as well as other required data toreproduce and constrain the spline by the corresponding surface. Splineprocessing unit 406 may then move to the next spline designated by user440 and further constrain additional splines by surface 414 or anothersurface. The resulting constrained splines and/or surfaces may berendered by rendered 418 and/or output to user 440 to inspect theresults.

Note that, in the context of describing disclosed embodiments, unlessotherwise specified, use of expressions regarding executableinstructions (also referred to as code, applications, agents, etc.)performing operations that “instructions” do not ordinarily performunaided (e.g., transmission of data, calculations, etc.) denotes thatthe instructions are being executed by a machine, thereby causing themachine to perform the specified operations.

As one skilled in the art will appreciate in light of this disclosure,certain embodiments may be capable of achieving certain advantages,including some or all of the following: (1) Techniques described andsuggested in the present disclosure improve the field of computing,especially the field of digital animation, by improving the computationtime to calculate complex curves that follow the surface of a digitalmodel. (2) Additionally, techniques described and suggested in thepresent disclosure improve the efficiency of computing systems by, sincethe computation time to calculate complex curves is reduced, to computeand render more complex and realistic models in digital animationsequences. (3) Moreover, techniques described and suggested in thepresent disclosure are necessarily rooted in computer technology inorder to overcome problems specifically arising with how to generatecomplex curves to simulate realistic fibers or other flexible objectswithin the computational and time constraints of producing a digitalanimation product.

For example, FIG. 5 illustrates the example visual content generationsystem 500 as might be used to generate imagery in the form of stillimages and/or video sequences of images. Visual content generationsystem 500 might generate imagery of live action scenes, computergenerated scenes, or a combination thereof. In a practical system, usersare provided with tools that allow them to specify, at high levels andlow levels where necessary, what is to go into that imagery. Forexample, a user might be an animation artist, such as user 440 depictedin FIG. 4 and might use visual content generation system 500 to captureinteraction between two human actors performing live on a sound stageand replace one of the human actors with a computer-generatedanthropomorphic non-human being that behaves in ways that mimic thereplaced human actor's movements and mannerisms, and then add in a thirdcomputer-generated character and background scene elements that arecomputer-generated, all in order to tell a desired story or generatedesired imagery.

Still images that are output by visual content generation system 500might be represented in computer memory as pixel arrays, such as atwo-dimensional array of pixel color values, each associated with apixel having a position in a two-dimensional image array. Pixel colorvalues might be represented by three or more (or fewer) color values perpixel, such as a red value, a green value, and a blue value (e.g., inRGB format). Dimensions of such a two-dimensional array of pixel colorvalues might correspond to a preferred and/or standard display scheme,such as 1920-pixel columns by 1280-pixel rows or 4096-pixel columns by2160-pixel rows, or some other resolution. Images might or might not bestored in a compressed format, but either way, a desired image may berepresented as a two-dimensional array of pixel color values. In anothervariation, images are represented by a pair of stereo images forthree-dimensional presentations and in other variations, an imageoutput, or a portion thereof, might represent three-dimensional imageryinstead of just two-dimensional views. In yet other embodiments, pixelvalues are data structures and a pixel value is associated with a pixeland can be a scalar value, a vector, or another data structureassociated with a corresponding pixel. That pixel value might includecolor values, or not, and might include depth values, alpha values,weight values, object identifiers or other pixel value components.

A stored video sequence might include a plurality of images such as thestill images described above, but where each image of the plurality ofimages has a place in a timing sequence and the stored video sequence isarranged so that when each image is displayed in order, at a timeindicated by the timing sequence, the display presents what appears tobe moving and/or changing imagery. In one representation, each image ofthe plurality of images is a video frame having a specified frame numberthat corresponds to an amount of time that would elapse from when avideo sequence begins playing until that specified frame is displayed. Aframe rate might be used to describe how many frames of the stored videosequence are displayed per unit time. Example video sequences mightinclude 24 frames per second (24 FPS), 50 FPS, 140 FPS, or other framerates. In some embodiments, frames are interlaced or otherwise presentedfor display, but for clarity of description, in some examples, it isassumed that a video frame has one specified display time, but othervariations might be contemplated.

One method of creating a video sequence is to simply use a video camerato record a live action scene, i.e., events that physically occur andcan be recorded by a video camera. The events being recorded can beevents to be interpreted as viewed (such as seeing two human actors talkto each other) and/or can include events to be interpreted differentlydue to clever camera operations (such as moving actors about a stage tomake one appear larger than the other despite the actors actually beingof similar build, or using miniature objects with other miniatureobjects so as to be interpreted as a scene containing life-sizedobjects).

Creating video sequences for story-telling or other purposes often callsfor scenes that cannot be created with live actors, such as a talkingtree, an anthropomorphic object, space battles, and the like. Such videosequences might be generated computationally rather than capturing lightfrom live scenes. In some instances, an entirety of a video sequencemight be generated computationally, as in the case of acomputer-animated feature film. In some video sequences, it is desirableto have some computer-generated imagery and some live action, perhapswith some careful merging of the two.

While computer-generated imagery might be creatable by manuallyspecifying each color value for each pixel in each frame, this is likelytoo tedious to be practical. As a result, a creator uses various toolsto specify the imagery at a higher level. As an example, a user might bean animation artist, such as user 440 depicted in FIG. 4 and mightspecify the positions in a scene space, such as a three-dimensionalcoordinate system, of objects and/or lighting, as well as a cameraviewpoint, and a camera view plane. From that, a rendering engine couldtake all of those as inputs, and compute each of the pixel color valuesin each of the frames. In another example, an artist specifies positionand movement of an articulated object having some specified texturerather than specifying the color of each pixel representing thatarticulated object in each frame.

In a specific example, a rendering engine performs ray tracing wherein apixel color value is determined by computing which objects lie along aray traced in the scene space from the camera viewpoint through a pointor portion of the camera view plane that corresponds to that pixel. Forexample, a camera view plane might be represented as a rectangle havinga position in the scene space that is divided into a grid correspondingto the pixels of the ultimate image to be generated, and if a raydefined by the camera viewpoint in the scene space and a given pixel inthat grid first intersects a solid, opaque, blue object, that givenpixel is assigned the color blue. Of course, for moderncomputer-generated imagery, determining pixel colors—and therebygenerating imagery—can be more complicated, as there are lightingissues, reflections, interpolations, and other considerations.

As illustrated in FIG. 5, a live action capture system 502 captures alive scene that plays out on a stage 504. Live action capture system 502is described herein in greater detail, but might include computerprocessing capabilities, image processing capabilities, one or moreprocessors, program code storage for storing program instructionsexecutable by the one or more processors, as well as user input devicesand user output devices, not all of which are shown.

In a specific live action capture system, cameras 506(1) and 506(2)capture the scene, while in some systems, there might be other sensor(s)508 that capture information from the live scene (e.g., infraredcameras, infrared sensors, motion capture (“mo-cap”) detectors, etc.).On stage 504, there might be human actors, animal actors, inanimateobjects, background objects, and possibly an object such as a greenscreen 510 that is designed to be captured in a live scene recording insuch a way that it is easily overlaid with computer-generated imagery.Stage 504 might also contain objects that serve as fiducials, such asfiducials 512(1)-(3), that might be used post-capture to determine wherean object was during capture. A live action scene might be illuminatedby one or more lights, such as an overhead light 514.

During or following the capture of a live action scene, live actioncapture system 502 might output live action footage to a live actionfootage storage 520. A live action processing system 522 might processlive action footage to generate data about that live action footage andstore that data into a live action metadata storage 524. Live actionprocessing system 522 might include computer processing capabilities,image processing capabilities, one or more processors, program codestorage for storing program instructions executable by the one or moreprocessors, as well as user input devices and user output devices, notall of which are shown. Live action processing system 522 might processlive action footage to determine boundaries of objects in a frame ormultiple frames, determine locations of objects in a live action scene,where a camera was relative to some action, distances between movingobjects and fiducials, etc. Where elements have sensors attached to themor are detected, the metadata might include location, color, andintensity of overhead light 514, as that might be useful inpost-processing to match computer-generated lighting on objects that arecomputer-generated and overlaid on the live action footage. Live actionprocessing system 522 might operate autonomously, perhaps based onpredetermined program instructions, to generate and output the liveaction metadata upon receiving and inputting the live action footage.The live action footage can be camera-captured data as well as data fromother sensors.

An animation creation system 530 is another part of visual contentgeneration system 500. Animation creation system 530 might includecomputer processing capabilities, image processing capabilities, one ormore processors, program code storage for storing program instructionsexecutable by the one or more processors, as well as user input devicesand user output devices, not all of which are shown. Animation creationsystem 530 might be used by animation artists, managers, and others tospecify details, perhaps programmatically and/or interactively, ofimagery to be generated. From user input and data from a database orother data source, indicated as a data store 532, animation creationsystem 530 might generate and output data representing objects (e.g., ahorse, a human, a ball, a teapot, a cloud, a light source, a texture,etc.) to an object storage 534, generate and output data representing ascene into a scene description storage 536, and/or generate and outputdata representing animation sequences to an animation sequence storage538.

Scene data might indicate locations of objects and other visualelements, values of their parameters, lighting, camera location, cameraview plane, and other details that a rendering engine 550 might use torender CGI imagery. For example, scene data might include the locationsof several articulated characters, background objects, lighting, etc.specified in a two-dimensional space, three-dimensional space, or otherdimensional space (such as a 2.5-dimensional space, three-quarterdimensions, pseudo-3D spaces, etc.) along with locations of a cameraviewpoint and view place from which to render imagery. For example,scene data might indicate that there is to be a red, fuzzy, talking dogin the right half of a video and a stationary tree in the left half ofthe video, all illuminated by a bright point light source that is aboveand behind the camera viewpoint. In some cases, the camera viewpoint isnot explicit, but can be determined from a viewing frustum. In the caseof imagery that is to be rendered to a rectangular view, the frustumwould be a truncated pyramid. Other shapes for a rendered view arepossible and the camera view plane could be different for differentshapes.

Animation creation system 530 might be interactive, allowing a user toread in animation sequences, scene descriptions, object details, etc.and edit those, possibly returning them to storage to update or replaceexisting data. As an example, an operator might read in objects fromobject storage into a baking processor 542 that would transform thoseobjects into simpler forms and return those to object storage 534 as newor different objects. For example, an operator might read in an objectthat has dozens of specified parameters (movable joints, color options,textures, etc.), select some values for those parameters and then save abaked object that is a simplified object with now fixed values for thoseparameters.

Rather than requiring user specification of each detail of a scene, datafrom data store 532 might be used to drive object presentation. Forexample, if an artist is creating an animation of a spaceship passingover the surface of the Earth, instead of manually drawing or specifyinga coastline, the artist might specify that animation creation system 530is to read data from data store 532 in a file containing coordinates ofEarth coastlines and generate background elements of a scene using thatcoastline data.

Animation sequence data might be in the form of time series of data forcontrol points of an object that has attributes that are controllable.For example, an object might be a humanoid character with limbs andjoints that are movable in manners similar to typical human movements.An artist can specify an animation sequence at a high level, such as“the left hand moves from location (X1, Y1, Z1) to (X2, Y2, Z2) overtime T1 to T2”, at a lower level (e.g., “move the elbow joint 2.5degrees per frame”) or even at a very high level (e.g., “character Ashould move, consistent with the laws of physics that are given for thisscene, from point P1 to point P2 along a specified path”).

Animation sequences in an animated scene might be specified by whathappens in a live action scene. An animation driver generator 544 mightread in live action metadata, such as data representing movements andpositions of body parts of a live actor during a live action scene.Animation driver generator 544 might generate corresponding animationparameters to be stored in animation sequence storage 538 for use inanimating a CGI object. This can be useful where a live action scene ofa human actor is captured while wearing mo-cap fiducials (e.g.,high-contrast markers outside actor clothing, high-visibility paint onactor skin, face, etc.) and the movement of those fiducials isdetermined by live action processing system 522. Animation drivergenerator 544 might convert that movement data into specifications ofhow joints of an articulated CGI character are to move over time.

A rendering engine 550 can read in animation sequences, scenedescriptions, and object details, as well as rendering engine controlinputs, such as a resolution selection and a set of renderingparameters. Resolution selection might be useful for an operator tocontrol a trade-off between speed of rendering and clarity of detail, asspeed might be more important than clarity for a movie maker to testsome interaction or direction, while clarity might be more importantthan speed for a movie maker to generate data that will be used forfinal prints of feature films to be distributed. Rendering engine 550might include computer processing capabilities, image processingcapabilities, one or more processors, program code storage for storingprogram instructions executable by the one or more processors, as wellas user input devices and user output devices, not all of which areshown.

Visual content generation system 500 can also include a merging system560 that merges live footage with animated content. The live footagemight be obtained and input by reading from live action footage storage520 to obtain live action footage, by reading from live action metadatastorage 524 to obtain details such as presumed segmentation in capturedimages segmenting objects in a live action scene from their background(perhaps aided by the fact that green screen 510 was part of the liveaction scene), and by obtaining CGI imagery from rendering engine 550.

A merging system 560 might also read data from rulesets formerging/combining storage 562. A very simple example of a rule in aruleset might be “obtain a full image including a two-dimensional pixelarray from live footage, obtain a full image including a two-dimensionalpixel array from rendering engine 550, and output an image where eachpixel is a corresponding pixel from rendering engine 550 when thecorresponding pixel in the live footage is a specific color of green,otherwise output a pixel value from the corresponding pixel in the livefootage.”

Merging system 560 might include computer processing capabilities, imageprocessing capabilities, one or more processors, program code storagefor storing program instructions executable by the one or moreprocessors, as well as user input devices and user output devices, notall of which are shown. Merging system 560 might operate autonomously,following programming instructions, or might have a user interface orprogrammatic interface over which an operator can control a mergingprocess. In some embodiments, an operator can specify parameter valuesto use in a merging process and/or might specify specific tweaks to bemade to an output of merging system 560, such as modifying boundaries ofsegmented objects, inserting blurs to smooth out imperfections, oradding other effects. Based on its inputs, merging system 560 can outputan image to be stored in a static image storage 570 and/or a sequence ofimages in the form of video to be stored in an animated/combined videostorage 572.

Thus, as described, visual content generation system 500 can be used togenerate video that combines live action with computer-generatedanimation using various components and tools, some of which aredescribed in more detail herein. While visual content generation system500 might be useful for such combinations, with suitable settings, itcan be used for outputting entirely live action footage or entirely CGIsequences. The code may also be provided and/or carried by a transitorycomputer readable medium, e.g., a transmission medium such as in theform of a signal transmitted over a network.

According to one embodiment, the techniques described herein areimplemented by one or more generalized computing systems programmed toperform the techniques pursuant to program instructions in firmware,memory, other storage, or a combination. Special-purpose computingdevices may be used, such as desktop computer systems, portable computersystems, handheld devices, networking devices or any other device thatincorporates hard-wired and/or program logic to implement thetechniques.

For example, FIG. 6 is a block diagram that illustrates a computersystem 600 upon which the computer systems of the systems describedherein and/or visual content generation system 500 (see FIG. 5) may beimplemented. Computer system 600 includes a bus 602 or othercommunication mechanism for communicating information, and a processor604 coupled with bus 602 for processing information. Processor 604 maybe, for example, a general-purpose microprocessor.

Computer system 600 also includes a main memory 606, such as arandom-access memory (RAM) or other dynamic storage device, coupled tobus 602 for storing information and instructions to be executed byprocessor 604. Main memory 606 may also be used for storing temporaryvariables or other intermediate information during execution ofinstructions to be executed by processor 604. Such instructions, whenstored in non-transitory storage media accessible to processor 604,render computer system 600 into a special-purpose machine that iscustomized to perform the operations specified in the instructions.

Computer system 600 further includes a read only memory (ROM) 608 orother static storage device coupled to bus 602 for storing staticinformation and instructions for processor 604. A storage device 610,such as a magnetic disk or optical disk, is provided and coupled to bus602 for storing information and instructions.

Computer system 600 may be coupled via bus 602 to a display 612, such asa computer monitor, for displaying information to a computer user. Aninput device 614, including alphanumeric and other keys, is coupled tobus 602 for communicating information and command selections toprocessor 604. Another type of user input device is a cursor control616, such as a mouse, a trackball, or cursor direction keys forcommunicating direction information and command selections to processor604 and for controlling cursor movement on display 612. This inputdevice typically has two degrees of freedom in two axes, a first axis(e.g., x) and a second axis (e.g., y), that allows the device to specifypositions in a plane.

Computer system 600 may implement the techniques described herein usingcustomized hard-wired logic, one or more ASICs or FPGAs, firmware and/orprogram logic which in combination with the computer system causes orprograms computer system 600 to be a special-purpose machine. Accordingto one embodiment, the techniques herein are performed by computersystem 600 in response to processor 604 executing one or more sequencesof one or more instructions contained in main memory 606. Suchinstructions may be read into main memory 606 from another storagemedium, such as storage device 610. Execution of the sequences ofinstructions contained in main memory 606 causes processor 604 toperform the process steps described herein. In alternative embodiments,hard-wired circuitry may be used in place of or in combination withsoftware instructions.

The term “storage media” as used herein refers to any non-transitorymedia that store data and/or instructions that cause a machine tooperation in a specific fashion. Such storage media may includenon-volatile media and/or volatile media. Non-volatile media includes,for example, optical or magnetic disks, such as storage device 610.Volatile media includes dynamic memory, such as main memory 606. Commonforms of storage media include, for example, a floppy disk, a flexibledisk, hard disk, solid state drive, magnetic tape, or any other magneticdata storage medium, a CD-ROM, any other optical data storage medium,any physical medium with patterns of holes, a RAM, a PROM, an EPROM, aFLASH-EPROM, NVRAM, any other memory chip or cartridge.

Storage media is distinct from but may be used in conjunction withtransmission media. Transmission media participates in transferringinformation between storage media. For example, transmission mediaincludes coaxial cables, copper wire, and fiber optics, including thewires that include bus 602. Transmission media can also take the form ofacoustic or light waves, such as those generated during radio-wave andinfra-red data communications.

Various forms of media may be involved in carrying one or more sequencesof one or more instructions to processor 604 for execution. For example,the instructions may initially be carried on a magnetic disk orsolid-state drive of a remote computer. The remote computer can load theinstructions into its dynamic memory and send the instructions over anetwork connection. A modem or network interface local to computersystem 600 can receive the data. Bus 602 carries the data to main memory606, from which processor 604 retrieves and executes the instructions.The instructions received by main memory 606 may optionally be stored onstorage device 610 either before or after execution by processor 604.

Computer system 600 also includes a communication interface 618 coupledto bus 602. Communication interface 618 provides a two-way datacommunication coupling to a network link 620 that is connected to alocal network 622. For example, communication interface 618 may be anetwork card, a modem, a cable modem, or a satellite modem to provide adata communication connection to a corresponding type of telephone lineor communications line. Wireless links may also be implemented. In anysuch implementation, communication interface 618 sends and receiveselectrical, electromagnetic, or optical signals that carry digital datastreams representing various types of information.

Network link 620 typically provides data communication through one ormore networks to other data devices. For example, network link 620 mayprovide a connection through local network 622 to a host computer 624 orto data equipment operated by an Internet Service Provider (ISP) 626.ISP 626 in turn provides data communication services through theworld-wide packet data communication network now commonly referred to asthe “Internet” 628. Local network 622 and Internet 628 both useelectrical, electromagnetic, or optical signals that carry digital datastreams. The signals through the various networks and the signals onnetwork link 620 and through communication interface 618, which carrythe digital data to and from computer system 600, are example forms oftransmission media.

Computer system 600 can send messages and receive data, includingprogram code, through the network(s), network link 620, andcommunication interface 618. In the Internet example, a server 630 mighttransmit a requested code for an application program through theInternet 628, ISP 626, local network 622, and communication interface618. The received code may be executed by processor 604 as it isreceived, and/or stored in storage device 610, or other non-volatilestorage for later execution.

Operations of processes described herein can be performed in anysuitable order unless otherwise indicated herein or otherwise clearlycontradicted by context. Processes described herein (or variationsand/or combinations thereof) may be performed under the control of oneor more computer systems configured with executable instructions and maybe implemented as code (e.g., executable instructions, one or morecomputer programs or one or more applications) executing collectively onone or more processors, by hardware or combinations thereof. The codemay be stored on a computer-readable storage medium, for example, in theform of a computer program comprising a plurality of instructionsexecutable by one or more processors. The computer-readable storagemedium may be non-transitory. The code may also be provided carried by atransitory computer readable medium e.g., a transmission medium such asin the form of a signal transmitted over a network.

Conjunctive language, such as phrases of the form “at least one of A, B,and C,” or “at least one of A, B and C,” unless specifically statedotherwise or otherwise clearly contradicted by context, is otherwiseunderstood with the context as used in general to present that an item,term, etc., may be either A or B or C, or any nonempty subset of the setof A and B and C. For instance, in the illustrative example of a sethaving three members, the conjunctive phrases “at least one of A, B, andC” and “at least one of A, B and C” refer to any of the following sets:{A}, {B}, {C}, {A, B}, {A, C}, {B, C}, {A, B, C}. Thus, such conjunctivelanguage is not generally intended to imply that certain embodimentsrequire at least one of A, at least one of B and at least one of C eachto be present.

The use of examples, or exemplary language (e.g., “such as”) providedherein, is intended merely to better illuminate embodiments of theinvention and does not pose a limitation on the scope of the inventionunless otherwise claimed. No language in the specification should beconstrued as indicating any non-claimed element as essential to thepractice of the invention.

In the foregoing specification, embodiments of the invention have beendescribed with reference to numerous specific details that may vary fromimplementation to implementation. The specification and drawings are,accordingly, to be regarded in an illustrative rather than a restrictivesense. The sole and exclusive indicator of the scope of the invention,and what is intended by the applicants to be the scope of the invention,is the literal and equivalent scope of the set of claims that issue fromthis application, in the specific form in which such claims issue,including any subsequent correction.

Further embodiments can be envisioned to one of ordinary skill in theart after reading this disclosure. In other embodiments, combinations orsub-combinations of the above-disclosed invention can be advantageouslymade. The example arrangements of components are shown for purposes ofillustration and combinations, additions, re-arrangements, and the likeare contemplated in alternative embodiments of the present invention.Thus, while the invention has been described with respect to exemplaryembodiments, one skilled in the art will recognize that numerousmodifications are possible.

For example, the processes described herein may be implemented usinghardware components, software components, and/or any combinationthereof. The specification and drawings are, accordingly, to be regardedin an illustrative rather than a restrictive sense. It will, however, beevident that various modifications and changes may be made thereuntowithout departing from the broader spirit and scope of the invention asset forth in the claims and that the invention is intended to cover allmodifications and equivalents within the scope of the following claims.

All references, including publications, patent applications, andpatents, cited herein are hereby incorporated by reference to the sameextent as if each reference were individually and specifically indicatedto be incorporated by reference and were set forth in its entiretyherein.

1. A computer-implemented method for use in a computer modeling system for generating a surface-aware spline relative to a first data structure of a curved surface, the computer-implemented method comprising: accessing the first data structure representing the curved surface in a three-dimensional (3D) space represented by the computer modeling system; receiving a first input of a starting point of the surface-aware spline in the 3D space relative to the curved surface; receiving a second input of an additional point for the surface-aware spline in the 3D space relative to the curved surface; generating a second data structure representing the surface-aware spline having at least a first vertex representing the starting point in the 3D space and a second vertex representing the additional point in the 3D space; projecting the first vertex onto the curved surface; projecting the second vertex onto the curved surface; determining new points corresponding to equal distance points for the first vertex and the second vertex aligned with the projected first vertex and the projected second vertex; from the new points and the curved surface, determining a bending rigid transformation for the surface-aware spline; adjusting the second data structure based on transforming the new points using the bending rigid transformation; receiving a user interface widget position in the 3D space relative to the curved surface; and determining heights between vertices and corresponding projection points of the surface-aware spline based on an attraction force between the first vertex and the user interface widget position.
 2. The computer-implemented method of claim 1, wherein the first input comprises a placement of the starting point on the curved surface.
 3. The computer-implemented method of claim 1, wherein adjusting the second data structure comprises computing a curvature-compensated operator for the second data structure using the new points and the bending rigid transformation.
 4. (canceled)
 5. The computer-implemented method of claim 1, wherein the second input comprises an end point of the surface-aware spline.
 6. The computer-implemented method of claim 5, further comprising: modifying a positioning of a point along the surface-aware spline between the starting point and the end point using one of a second position of the point or an attraction point for the point; and further adjusting the second data structure based on the modifying.
 7. The computer-implemented method of claim 1, further comprising determining a normal plane to the curved surface at the starting point in a 3D space; determining a starting point tangent plane that runs through the starting point based at least on the normal plane; and further adjusting the second data structure of the surface-aware spline by the starting point tangent plane.
 8. The computer-implemented method of claim 1, further comprising: obtaining an additional point tangent plane that runs through the additional point based at least on a constraint provided with the second input and a normal of a curvature of the curved surface; and further adjusting the second data structure of the surface-aware spline by the additional point tangent plane.
 9. The computer-implemented method of claim 1, further comprising: receiving a velocity value assigned to the starting point as an additional initial constraint on the first input; and further adjusting the second data structure of the surface-aware spline by the velocity value.
 10. A computer system for use in a computer modeling system for generating a surface-aware spline relative to a first data structure of a curved surface, the computer system comprising: at least one processor; and a computer-readable medium storing instructions, which when executed by the at least one processor, causes the computer system to perform operations comprising: accessing the first data structure representing the curved surface in a three-dimensional (3D) space represented by the computer modeling system; receiving a first input of a starting point of the surface-aware spline in the 3D space relative to the curved surface; receiving a second input of an additional point for the surface-aware spline in the 3D space relative to the curved surface; generating a second data structure representing the surface-aware spline having at least a first vertex representing the starting point in the 3D space and a second vertex representing the additional point in the 3D space; projecting the first vertex onto the curved surface; projecting the second vertex onto the curved surface; determining new points corresponding to equal distance points for the first vertex and the second vertex aligned with the projected first vertex and the projected second vertex; from the new points and the curved surface, determining a bending rigid transformation for the surface-aware spline; adjusting the second data structure based on transforming the new points using the bending rigid transformation; receiving a user interface widget position in the 3D space relative to the curved surface; and determining heights between vertices and corresponding projection points of the surface-aware spline based on an attraction force between the first vertex and the user interface widget position.
 11. The computer system of claim 10, wherein the first input comprises a placement of the starting point on the curved surface.
 12. The computer system of claim 10, wherein adjusting the second data structure comprises computing a curvature-compensated operator for the second data structure using the new points and the bending rigid transformation.
 13. (canceled)
 14. The computer system of claim 10, wherein the second input comprises an end point of the surface-aware spline.
 15. The computer system of claim 14, wherein the operations further comprise: modifying a positioning of a point along the surface-aware spline between the starting point and the end point using one of a second position of the point or an attraction point for the point; and further adjusting the second data structure based on the modifying.
 16. The computer system of claim 10, wherein the operations further comprise: determining a normal plane to the curved surface at the starting point in a 3D space; determining a starting point tangent plane that runs through the starting point based at least on the normal plane; and further adjusting the second data structure of the surface-aware spline by the starting point tangent plane.
 17. The computer system of claim 10, wherein the operations further comprise: obtaining an additional point tangent plane that runs through the additional point based at least on a constraint provided with the second input and a normal of a curvature of the curved surface; and further adjusting the second data structure of the surface-aware spline by the additional point tangent plane.
 18. The computer system of claim 10, wherein the operations further comprise: receiving a velocity value assigned to the starting point as an additional initial constraint on the first input; and further adjusting the second data structure of the surface-aware spline by the velocity value.
 19. A non-transitory processor-readable medium storing a plurality of processor-executable instructions for use in a computer modeling system for generating a surface-aware spline relative to a first data structure of a curved surface, the plurality of processor-executable instructions being executed by a processor to perform operations comprising: accessing the first data structure representing the curved surface in a three-dimensional (3D) space represented by the computer modeling system; receiving a first input of a starting point of the surface-aware spline in the 3D space relative to the curved surface; receiving a second input of an additional point for the surface-aware spline in the 3D space relative to the curved surface; generating a second data structure representing the surface-aware spline having at least a first vertex representing the starting point in the 3D space and a second vertex representing the additional point in the 3D space; projecting the first vertex onto the curved surface; projecting the second vertex onto the curved surface; determining new points corresponding to equal distance points for the first vertex and the second vertex aligned with the projected first vertex and the projected second vertex; from the new points and the curved surface, determining a bending rigid transformation for the surface-aware spline; adjusting the second data structure based on transforming the new points using the bending rigid transformation; receiving a user interface widget position in the 3D space relative to the curved surface; and determining heights between vertices and corresponding projection points of the surface-aware spline based on an attraction force between the first vertex and the user interface widget position.
 20. The non-transitory processor-readable medium of claim 19, wherein adjusting the second data structure comprises computing a curvature-compensated operator for the second data structure using the new points and the bending rigid transformation.
 21. The computer-implemented method of claim 1, further comprising: further adjusting the second data structure based on the heights determined from the attraction force between the first vertex and the user interface widget position.
 22. The computer system of claim 10, wherein the operations further comprise: further adjusting the second data structure based on the heights determined from the attraction force between the first vertex and the user interface widget position. 